Gesture interfaces based on inertial sensors such as accelerometers and gyroscopes embedded in small form factor devices (e.g. a sensor-enabled handheld device or wrist-watch) are becoming increasingly common in user devices such as smart phones, remote controllers and game consoles.
In the mobile space, gesture interaction is an attractive alternative to traditional interfaces because it does not contain the shrinking of the form factor of traditional input devices such as a keyboard and mouse and screen. In addition, gesture interaction is more supportive of mobility, as users can easily do subtle gestures as they walk around or drive.
“Dynamic 3D gestures” are based on atomic movements of a user using inertial sensors such as micro-electromechanical system (MEMS) based accelerometers and gyroscopes. Statistical recognition algorithms, such as Hidden Markov Model algorithms (HMM), are widely used for gesture and speech recognition and many other machine learning tasks. Research has shown HMM to be extremely effective for recognizing complex gestures and enabling rich gesture input vocabularies.
Several challenges arise when using HMM for gesture recognition in mobile devices. HMM is computationally demanding (e.g., O(num_of_samples*HMM_num_states^2)). Furthermore, to obtain highly accurate results, continuous Gaussian Mixtures are usually employed in HMM's output probabilities, whose probability density function evaluation is computationally expensive. Matching an incoming signal with several models (typically one per trained gesture) for finding the best match (e.g. using Viterbi decoding in HMM) is also computationally intensive.
Low latency requirements of mobile devices pose a problem in real time gesture recognition on resource constrained devices, especially when using techniques for improving accuracy, e.g. changing gesture “grammar” or statistical models on the fly.
Additionally, for a high level of usability, gestures should be easy to use. Common techniques based on push/release buttons for gesture spotting should be avoided. Inexact interaction based only on shake/whack gestures limits the user experience. Finally, using a simple and easily recognizable gesture to trigger gesture recognition would be cumbersome in complex and sustained gesture-based user interactions.
A straight forward approach to mitigate these issues would be to run continuous HMM (CHMM) for gesture spotting and recognition. However this will trigger many false positives and is not efficient with regards to power consumption and processing.
Current gesture interfaces also typically choose one single algorithm to recognize all the gestures, based on the type of expected user gestures. For example, dynamic movement tracking is typically employed by smart-phone applications, while continuous tracking may be used in motion detection gaming consoles. Thus, gesture recognition devices are typically configured to recognize and process only a specific type of gesture.
Descriptions of certain details and implementations follow, including a description of the figures, which may depict some or all of the embodiments described below, as well as discussing other potential embodiments or implementations of the inventive concepts presented herein. An overview of embodiments of the invention is provided below, followed by a more detailed description with reference to the drawings.